BRO05424 Improving Standardized Reading Comprehension: The Role of Question-Answering

This paper provides empirical evidence that effective instruction in questionanswering leads to statistically significant improvements in reading comprehension, when compared to regular classroom reading instruction. The presentation reports both the features of intervention materials and the differences in reading instruction between a treatment and control group that contributed to differences in posttest treatment group performance. The study involved a quasi-experimental, pretestposttest design that targeted students enrolled in regular Year 5 classrooms across three schools. There were no statistically significant pretest differences between the treatment groups. Classroom teachers implemented the intervention with their classes over a ten week period. Comparisons were made between students who completed their regular classroom reading program and students who completed the intervention. Statistical analyses used multilevel modelling to ensure that adjustments were made for potential differences at the treatment group level and at the class level. Posttest comparisons on both a standardised reading comprehension measure and researcherdevised question-answering measures significantly favoured the intervention group. This presentation outlines the theoretical foundation and methodology for effective classroom instruction in question-answering. The potential future applications of this instructional technology to a range of complex cognitive skills are discussed. The impact of reading comprehension on our daily lives has never been as crucial as in our modern society today. Individuals use literacy skills to communicate relationships between complex concepts and knowledge. Considerable literature has documented researchers on the state of reading (Kamil, Mosenthal, Pearson, & Barr, 2000; National Institute of Child Health and Human Development [NICHHD], 2000a) and reading comprehension (Muth, 1990; Pearson & Johnson, 1978; Pressley & Afflerbach, 1995). Research reviews have focussed specifically on reading comprehension and its instruction (Dole, Duffy, Roehler, & Pearson, 1991; Fielding & Pearson, 1994; Pearson & Fielding, 1991; Pressley, Brown, El-Dinary & Afflerbach, 1995; Rosenshine & Meister, 1994) and on results for specific student populations (Gersten, Fuchs, Williams, & Baker, 2001; Mastropieri & Scruggs, 1997; Weisberg, 1988). However, effective classroom instructional programs for reading comprehension are yet to be identified. Reading is a complex, cognitive process: “...a whole complex system of skills and knowledge... knowledge and activities in visually recognising individual printed words are useless in and of themselves...” (Adams, 1990, p.3). Of particular relevance to this study were Adams additional comments that such decoding processes should be “guided and received by complementary knowledge and activities of language comprehension” (page 3). She implicitly supported the “simple view of reading” which outlined that reading was basically the product of decoding and comprehension processes (Hoover & Gough, 1990, p. 127; Hoover & Tunmer, 1993). Taking this view further, the National Reading Panel identified four key components of reading skills: Phonemic skills, vocabulary, reading fluency and comprehension (National Institute of Child Health and Human Development [NICHHD], 2000a). Identification of these four key components was primarily based on research on beginning reading and the prevention of difficulties in learning to read (Abbott, Walton, & Greenwood, 2002; Denton, Vaughn & Fletcher, 2003; Elliott, Lee & Tollafson, 2001; Good III, Simmons & Kameenui, 2001; Kaminski & Good III, 1998; Snow, Burns, & Griffin, 1998). More recently, empirical data and descriptive analyses reflecting these four components has outlined different types of readers based on patterns of strength and weakness across reading accuracy, reading fluency, vocabulary knowledge and question-answering (Valencia & Buly, 2004). In addition, a complex picture of neurological functioning relevant to reading has emerged (Johnson, Hetzel & Collins, 2002). In the current study, these four components of reading were reflected in the measures used to determine the efficacy of the intervention program. These included standardised reading comprehension and vocabulary, oral reading fluency and written question-answering measures. More specific to this study, researchers have called for instructional reforms in reading comprehension for decades (Ares & Peercy, 2003; Biemiller, 1994; Durkin, 1978-9; Schmidt, Rozendal, & Greenman, 2002; Simons, 1971; Thurlow, Ysseldyke, Wotruba, & Algozzine, 1993). However, two key limitations have persisted. Firstly, researchers have often failed to identify effective reading comprehension teaching strategies in sufficient detail to serve as an instructional program for classroom teachers. Instead, general methods of implementation have been suggested using terms such as “explicit instruction” (Pearson & Dole, 1987, p. 151), “thinking aloud” (Kucan & Beck, 1996, p. 259) and “direct instruction” (Carnine, Silbert, & Kameenui, 1997, p.1). This has met with considerable difficulty by teachers in attempting to translate these recommendations into instructional programs for use in classrooms. Also, researchers have only loosely defined the comprehension curriculum in terms of specific comprehension (Dole et al., 1991; Fielding & Pearson, 1994; Guszak, 1967; McNeil, 1987; Pearson & Fielding, 1991; Pressley, El-Dinary et al., 1992; Rosenshine & Meister, 1994). The National Reading Panel reviewed extant research for the purpose of improving classroom instruction in America (NICHHD, 2000a). Instruction in seven categories of specific reading comprehension skills, including question-answering instruction were reviewed. However, recommendations emanating from this report remained limited to general teaching strategies for classroom teachers. There has been little consideration of the selection of teaching examples and the crucial role of student materials for improving classroom instruction in reading comprehension: that is, a specific instructional program for classrooms (Gersten et al., 2001). Historically, it would seem that teachers have been provided with suggestions for general methods for teaching comprehension and still have not been provided with either a clearly defined instructional program for the teaching of specific reading comprehension skills nor with appropriate classroom materials. Reading comprehension is a cognitive process by nature. Recent theoretical advances, using information processing models, offer some promise for improving the efficacy of instructional interventions in reading comprehension research (Bransford, Brown, & Cocking, 2000; Coltheart, Rastle, Perry, Langdon, & Ziegler, 2001; Donovan, Bransford, & Pellegrino, 2001; Gordon, Hendrick, & Johnson, 2001; Shavelson & Towne, 2002). Information processing models utilise analogies between computer systems and human cognition. These models provide a theoretical basis for detailed analysis and simulation of complex cognitive tasks, including those found in classrooms and workplaces (Baddeley, Aggleton, & Conway, 2001; Kintsch, 1998; Miyake & Shah, 1999b; Newell, 1990; van Merrienboer & Paas, 2003). For example information processing models have providing insights into the cognitive processes used in decoding (Coltheart et al., 2001). La Berge and Samuels (1974) applied the foundational concepts of automaticity and capacity limitations to reading. However, their model predominantly outlined the decoding processes of letter sounds. The model was of limited application to the current study where the focus was on question-answering as one reading comprehension skill. To date, there does not appear to have been a specific application of information processing models to the design of classroom materials that was focussed on how to answer questions. Information processing models define two broad types of knowledge: declarative facts and procedural knowledge (Aitkenhead & Slack, 1985; Anderson, 1993; Hasselbring, Goin, & Bransford, 1988; Sieck & Yates, 2001; Sorace, Heycock, & Shillcock, 1999). Declarative facts are stored and retrieved more accurately and effortlessly depending on the strength of the relationship between a stimulus and a response (Hasselbring et al., 1988) or on the number of opportunities for practice with a particular stimulus (Logan, 1988). In the present study, declarative knowledge includes the meanings of words in questions and the types of questions taught. Procedural knowledge is defined as knowledge of sequences of steps in a strategy (Howell & Nolet, 2000; Pellegrino & Goldman, 1987). In the current study, procedural knowledge includes the strategy steps used for question-answering. Over time and with practice, declarative and procedural knowledge practised and transformed into complex skill levels of expert performance (Bransford et al., 2000; De Corte, 2003; De Corte, Verschaffel, Enwistle & van Merrienboer, 2003; Engelmann & Carnine, 1982). Skill development is a gradual process that takes time and involves changes to efficient strategies (Anderson, 1982, 2002; Goldman, Mertz, & Pellegrino, 1989; Strayer & Kramer, 1994). The development of such expertise can take a period of ten years or longer (Ericsson, Krampe, & Tesch-Romer, 1993). Everyday examples of such complex skills might include playing musical instruments, driving a car, playing a sport, and reading (Bransford, et al., 2000; Chaffin & Imreh, 2002; Ericsson, et al., 1993; Langan-Fox, Armstrong, Balvin & Anglim, 2002; Proctor & Dutta, 1995). Attention and working memory are two broad constructs in information processing models that may impact on skill development. The design of the current intervention program first focuses on selecting examples with specific features that direct attention to features critical to concept learning (Engelmann, 1980; Engelmann & Carnine, 1982; Howell & Nolet, 2000; Thorley, 1987; van Merrienboer & Paas, 2003). The intervention program then gradually increases the difficulty of examples and their instructional context across lessons in order to take account of the working memory limitations in the completion of question-answering (Sweller, van Merrienboer, & Paas, 1998; van Merrienboer & Paas, 2003). Researchers have been able to decrease cognitive load by simplifying tasks and increasing task difficulty over time. Sweller and his colleagues have emphasised the importance of completing simple “part-task practice” as part of limiting the effects of working memory (van Merrienboer & Paas, 2003, p. 11). Completed examples have been shown to reduce working memory limitations and provide support to learners (van Merrienboer & Paas, 2003). These features are incorporated into the intervention program in the current study. Question-answering is both a common indicator of reading comprehension and integral to our daily lives. Research has confirmed the importance of questionanswering to classroom functioning, and specifically to reading comprehension (Andre, 1987; Armbruster, 1992; Beck, McKeown, Hamilton, & Kucan, 1997; Cazden, 1988; Guszak, 1967; Rickards, 1979; Weedman & Weedman, 2001). As such, it could be readily taught within a classroom instructional program. Despite this wealth of discussion and research, effective instructional programs for questionanswering have not been evident. Durkin (1978-9) reported classroom teaching practices that predominantly involved repeated teacher assessments rather than instruction on how to comprehend the question. By clearly linking questions with answers using passages of text, a small body of research has provided some insights for how students might approach the task of answering questions (Pearson & Johnson, 1978; Raphael, 1982). Pearson and Johnson’s (1978) taxonomy of question-answer relations was based on reading theories that viewed text reading as an interactive process involving the text and the reader. Raphael’s (1982) interpretation of Pearson and Johnson’s (1978) taxonomy was utilised for the present study. This interpretation involved “Right There”, “Think and Search” and “On My Own” question-answer relationships (p. 188). Selection of Raphael’s (1982) interpretation was based on age-appropriateness of the language for the participants in the study, use of terminology that indicated processing steps within the definitions, and the ease of translation of the “Right There” question definition for teaching particular examples that used one sentence in the text (see Appendix A, for definitions as used in the intervention). Studies by Raphael and her colleagues (Raphael, 1982, 1984; Raphael & McKinney, 1983; Raphael & Wonnacott, 1985) had reported improvements in researcher developed measures but no corresponding improvements in standardised reading comprehension. In reading comprehension strategy instruction, some “powerful learning environments” (van Merrienboer & Paas, 2003, p.3) have documented significant gains in student performance, notably excluding standardised reading comprehension measures (De Corte, Verschaffel & Van de Ven, 2001; De Corte, et al., 2003). Previous reading comprehension research, including question-answering research, has been threatened by serious methodological flaws (Lysynchuk, Pressley, d’Ailly, Smith & Cake, 1989). Lysynchuk et al. (1989) reported weaknesses including questionable validity and reliability, limited empirical data, small sample sizes, specific types of participants and insufficient details of methodology to enable replication. The quasi-experimental design of the current study, reporting valid, reliable measures with inter-rater reliability and integrity of implementation data, addresses many methodological weaknesses of previous studies. The primary purpose of the current study is to determine the effectiveness of a theoretically designed question-answering program to enhance standardised reading comprehension, and question-answering performance of Year 5 students using complex, statistical analyses. A secondary purpose of the current study is to develop an empirically validated question-answering program that can be readily implemented by classroom teachers. The specific hypotheses were that students who completed the question-answering intervention will demonstrate statistically significantly higher scores on a standardised measure of reading comprehension P.A.T.Comprehension) (Australian Council for Educational Research, 1986) and on measures of written question-answering on a narrative and a factual passage than Year 5 students who completed regular classroom reading programs.